Vision Transformers (ViTs) have become a dominant paradigm for visual representation learning with self-attention operators. Although these operators provide flexibility to the model with their adjustable attention kernels, they suffer from inherent limitations: (1) the attention kernel is not discriminative enough, resulting in high redundancy of the ViT layers, and (2) the complexity in computation and memory is quadratic in the sequence length. In this paper, we propose a novel attention operator, called lightweight structure-aware attention (LiSA), which has a better representation power with log-linear complexity. Our operator learns structural patterns by using a set of relative position embeddings (RPEs). To achieve log-linear complexity, the RPEs are approximated with fast Fourier transforms. Our experiments and ablation studies demonstrate that ViTs based on the proposed operator outperform self-attention and other existing operators, achieving state-of-the-art results on ImageNet, and competitive results on other visual understanding benchmarks such as COCO and Something-Something-V2. The source code of our approach will be released online.

We introduce hp-greedy, a refinement approach for building gravitational wave surrogates as an extension of the standard reduced basis framework. Our proposal is data-driven, with a domain decomposition of the parameter space, local reduced basis, and a binary tree as the resulting structure, which are obtained in an automated way. When compared to the standard global reduced basis approach, the numerical simulations of our proposal show three salient features: i) representations of lower dimension with no loss of accuracy, ii) a significantly higher accuracy for a fixed maximum dimensionality of the basis, in some cases by orders of magnitude, and iii) results that depend on the reduced basis seed choice used by the refinement algorithm. We first illustrate the key parts of our approach with a toy model and then present a more realistic use case of gravitational waves emitted by the collision of two spinning, non-precessing black holes. We discuss performance aspects of hp-greedy, such as overfitting with respect to the depth of the tree structure, and other hyperparameter dependences. As two direct applications of the proposed hp-greedy refinement, we envision: i) a further acceleration of statistical inference, which might be complementary to focused reduced-order quadratures, and ii) the search of gravitational waves through clustering and nearest neighbors.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

本文介绍了BRL/PISA/IIT（BPI）SOFTHAND：单个执行器驱动的，低成本，3D打印，肌腱驱动的机器人手，可用于执行一系列掌握任务。基于PISA/IIT SOFTHAND的自适应协同作用，我们设计了一种新的关节系统和肌腱路由，以促进软化和适应性的协同作用，这有助于我们平衡手的耐用性，负担能力和握住手的性能。这项工作的重点在于该杂种的设计，仿真，协同作用和抓握测试。新颖的小块是根据连锁，齿轮对和几何约束机制设计和印刷的，可以应用于大多数肌腱驱动的机器人手。我们表明，机器人手可以成功地掌握和提起各种目标对象并适应复杂的几何形状，从而反映了软化和适应性协同的成功采用。我们打算为手的设计开放源，以便可以在家用3D打印机上廉价地构建。有关更多详细信息：https：//sites.google.com/view/bpi-softhandtactile-group-bri/brlpisaiit-softhand-design

我们概括了模型预测路径积分控制（MPPI）的推导，以允许对照序列中的对照组进行单个关节分布。这种改革允许实施自适应重要性采样（AIS）算法，以在最初的重要性采样步骤中实施，同时仍保持MPPI的好处，例如使用任意系统动态和成本功能。在模拟环境中证明了通过在每个控制步骤中集成AIS来优化建议分布的好处，包括控制轨道周围的多辆车。新算法比MPPI更有效地样品，可以通过更少的样品实现更好的性能。随着动作空间的维度的增加，这种性能差异会增长。模拟的结果表明，新算法可以用作任何时间算法，从而增加了每次迭代的控制值与依赖大量样品的算法。

The ability to generate dynamic walking in real-time for bipedal robots with input constraints and underactuation has the potential to enable locomotion in dynamic, complex and unstructured environments. Yet, the high-dimensional nature of bipedal robots has limited the use of full-order rigid body dynamics to gaits which are synthesized offline and then tracked online. In this work we develop an online nonlinear model predictive control approach that leverages the full-order dynamics to realize diverse walking behaviors. Additionally, this approach can be coupled with gaits synthesized offline via a desired reference to enable a shorter prediction horizon and rapid online re-planning, bridging the gap between online reactive control and offline gait planning. We demonstrate the proposed method, both with and without an offline gait, on the planar robot AMBER-3M in simulation and on hardware.

生产精确的天气预报和不确定的不确定性的可靠量化是一个开放的科学挑战。到目前为止，集团预测是最成功的方法，以产生相关预测的方法以及估计其不确定性。集合预测的主要局限性是高计算成本，难以捕获和量化不同的不确定性来源，特别是与模型误差相关的源。在这项工作中，进行概念证据模型实验，以检查培训的ANN的性能，以预测系统的校正状态和使用单个确定性预测作为输入的状态不确定性。我们比较不同的培训策略：一个基于使用集合预测的平均值和传播作为目标的直接培训，另一个依赖于使用确定性预测作为目标的决定性预测，其中来自数据隐含地学习不确定性。对于最后一种方法，提出和评估了两个替代损失函数，基于数据观察似然和基于误差的本地估计来评估另一个丢失功能。在不同的交货时间和方案中检查网络的性能，在没有模型错误的情况下。使用Lorenz'96模型的实验表明，ANNS能够模拟集合预测的一些属性，如最不可预测模式的过滤和预测不确定性的状态相关量化。此外，ANNS提供了在模型误差存在下的预测不确定性的可靠估计。

灵巧的操纵仍然是机器人技术中的一个空缺问题。为了协调研究界为解决这个问题的努力，我们提出了共同的基准。我们设计和构建了机器人平台，该平台托管在MPI上供智能系统托管，可以远程访问。每个平台由三个能够敏捷物体操纵的机器人手指组成。用户能够通过提交自动执行的代码（类似于计算群集）来远程控制平台。使用此设置，i）我们举办机器人竞赛，来自世界任何地方的团队访问我们的平台以应对具有挑战性的任务ii）我们发布了在这些比赛中收集的数据集（包括数百个机器人小时），而我们为研究人员提供了访问自己项目的这些平台。

癫痫发作是最重要的神经障碍之一，其早期诊断将有助于临床医生为患者提供准确的治疗方法。脑电图（EEG）信号广泛用于癫痫癫痫发作检测，其提供了关于大脑功能的实质性信息的专家。本文介绍了采用模糊理论和深层学习技术的新型诊断程序。所提出的方法在Bonn大学数据集上进行了评估，具有六个分类组合以及弗赖堡数据集。可以使用可调谐Q小波变换（TQWT）来将EEG信号分解为不同的子带。在特征提取步骤中，从TQWT的不同子带计算了13个不同的模糊熵，并且计算它们的计算复杂性以帮助研究人员选择各种任务的最佳集合。在下文中，采用具有六层的AutoEncoder（AE）用于减少维数。最后，标准自适应神经模糊推理系统（ANFIS）以及其具有蚱蜢优化算法（ANFIS-GOA），粒子群优化（ANFIS-PSO）和育种群优化（ANFIS-BS）方法的变体分类。使用我们所提出的方法，ANFIS-BS方法在弗赖堡数据集上分为两类分为两类和准确度，在两类分类中获得99.46％的准确性，以及弗赖堡数据集的99.28％，达到最先进的两个人的表演。

We demonstrate a proof-of-concept of a large language model conducting corporate lobbying related activities. We use an autoregressive large language model (OpenAI's text-davinci-003) to determine if proposed U.S. Congressional bills are relevant to specific public companies and provide explanations and confidence levels. For the bills the model deems as relevant, the model drafts a letter to the sponsor of the bill in an attempt to persuade the congressperson to make changes to the proposed legislation. We use hundreds of ground-truth labels of the relevance of a bill to a company to benchmark the performance of the model, which outperforms the baseline of predicting the most common outcome of irrelevance. However, we test the ability to determine the relevance of a bill with the previous OpenAI GPT-3 model (text-davinci-002), which was state-of-the-art on many language tasks until text-davinci-003 was released on November 28, 2022. The performance of text-davinci-002 is worse than simply always predicting that a bill is irrelevant to a company. These results suggest that, as large language models continue to improve core natural language understanding capabilities, performance on corporate lobbying related tasks will continue to improve. We then discuss why this could be problematic for societal-AI alignment.